Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Limit maximum HTTP Range header to fileSize #198

Merged
merged 1 commit into from
Feb 4, 2021
Merged

Limit maximum HTTP Range header to fileSize #198

merged 1 commit into from
Feb 4, 2021

Conversation

aloisklink
Copy link
Contributor

Some HTTP servers do not support Range requests when the Range header is longer than the actual length of the file. Examples of servers that don't support it are Aliyun OSS (see #196 and examples below, and it looks like node-static also doesn't support it).

I was reading through RFC 7233: Range Requests but it doesn't seem to say anything about what servers should do when the byte range is longer than the file range. So I guess Aliyun OSS is technically in the spec when they don't support it, and it's something we need to handle on the client side.

Fixes #196

Example

The length of the following file is 44515988 bytes.

Returns HTTP 200

The following HTTP range request returns HTTP 200 and downloads the entire file since the end of the range is past the end of the file.

me@home:~$ curl -v -r 44498944-44564479 https://debug-image-bucket.oss-eu-west-1.aliyuncs.com/tile_1_2.tif > range.tif
> GET /tile_1_2.tif HTTP/1.1
> Host: debug-image-bucket.oss-eu-west-1.aliyuncs.com
> Range: bytes=44498944-44564479
> User-Agent: curl/7.68.0
> Accept: */*
> 
{ [5 bytes data]
< HTTP/1.1 200 OK
< Server: AliyunOSS
< Content-Type: image/tiff
< Content-Length: 44515988

Returns HTTP 206

The following command returns HTTP 206 and works correctly.

me@home:~$ curl -v -r 44498944-44515987 https://debug-image-bucket.oss-eu-west-1.aliyuncs.com/tile_1_2.tif > range.tif
> GET /tile_1_2.tif HTTP/1.1
> Host: debug-image-bucket.oss-eu-west-1.aliyuncs.com
> Range: bytes=44498944-44515987
> User-Agent: curl/7.68.0
> Accept: */*
> 
{ [5 bytes data]
< HTTP/1.1 206 Partial Content
< Server: AliyunOSS
< Content-Type: image/tiff
< Content-Length: 17044
< Content-Range: bytes 44498944-44515987/44515988

@aloisklink
Limit maximum HTTP Range header to fileSize
21c197a 
Some HTTP servers (e.g. Aliyun OSS) do not support Range requests
when the Range header is longer than the actual length of the file.
@constantinius
Copy link
Member

All HTTP servers I interacted with did not have this issue, so I never had troubles with that. Your solution makes sense and can't really hurt, I guess.

Thanks!

@constantinius constantinius merged commit e9817ff into geotiffjs:master Feb 4, 2021
@aloisklink aloisklink deleted the limit-range-to-file-size branch February 4, 2021 13:46
@aloisklink
Copy link
Contributor Author

All HTTP servers I interacted with did not have this issue, so I never had troubles with that. Your solution makes sense and can't really hurt, I guess.

I know! Aliyun OSS is an outlier. While debugging, I saw that node-static also has the same issue cloudhead/node-static#209, so maybe that's their back-end library.

Thanks to you too for making this amazing library open-source 😄!

@constantinius constantinius mentioned this pull request Feb 4, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Retrieving data with fromUrl gets wrong data from Alibaba OSS
2 participants
@aloisklink @constantinius